{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p align=\"center\">\n",
    "  <img src=\"https://github.com/wisupai/e2m/blob/main/docs/images/wisup_e2m_banner.jpg?raw=true\" width=\"100%\" alt=\"wisup_e2m Logo\">\n",
    "</p>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 👏🏻 Welcome to E2M (Everything2Markdown) 👏🏻\n",
    "\n",
    "## 📖 Introduction\n",
    "\n",
    "E2M is a tool that converts various content into Markdown format, supporting multiple input formats:\n",
    "\n",
    "- **Text**\n",
    "  - doc\n",
    "  - docx\n",
    "  - epub\n",
    "  - html\n",
    "  - htm\n",
    "  - pdf (Note: includes plain text, text + image, and image-only PDF files)\n",
    "  - ppt\n",
    "  - pptx\n",
    "- **Links**\n",
    "  - url\n",
    "- **Audio**\n",
    "  - mp3\n",
    "  - m4a\n",
    "- **Video**\n",
    "  - mp4 (in progress)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 🚀 Quick Start\n",
    "\n",
    "### 🔧 Installation\n",
    "\n",
    "```bash\n",
    "pip install wisup_e2m\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Core Feature 1: Parser\n",
    "\n",
    "The purpose of the Parser is to extract text or images from various types of files. Since the main inputs for large models are text and images, the parser serves as a preprocessing step before running the Converter.\n",
    "\n",
    "The data format returned after parsing is `E2MParsedData`:\n",
    "\n",
    "```python\n",
    "class E2MParsedData(BaseModel):\n",
    "    text: Optional[str] = Field(None, description=\"Parsed text\")\n",
    "    images: Optional[List[str]] = Field([], description=\"Parsed image paths\")\n",
    "    attached_images: Optional[List[str]] = Field(\n",
    "        [], description=\"Attached image paths, like 1_0.png, 1_1.png, etc.\"\n",
    "    )\n",
    "    attached_images_map: Optional[Dict[str, List[str]]] = Field(\n",
    "        {},\n",
    "        description=\"Attached image paths map, like {1.png: ['/path/to/1_0.png'], 2.png: ['/path/to/2_1.png']}, only available for layout detection.\",\n",
    "    )\n",
    "    metadata: Optional[List[Any] | Dict[str, Any]] = Field(\n",
    "        {}, description=\"Metadata of the parsed data, including engine, etc.\"\n",
    "    )\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### URL Parser"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from wisup_e2m import UrlParser\n",
    "\n",
    "url = \"https://www.osar.fr/notes/justintonation\"\n",
    "parser = UrlParser(engine=\"jina\") # url engines: jina, firecrawl, unstructured"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "url_data = parser.parse(url)\n",
    "print(url_data.text)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### PDF Parser"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Using the Marker Engine will download models from the Hugging Face Hub and use them for parsing.\n",
    "# If you encounter network issues, try running the following code:\n",
    "\n",
    "import os\n",
    "os.environ['CURL_CA_BUNDLE'] = ''\n",
    "os.environ['HF_ENDPOINT'] = 'https://hf-mirror.com'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from wisup_e2m import PdfParser\n",
    "\n",
    "pdf_path = \"./test.pdf\"\n",
    "parser = PdfParser(engine=\"marker\") # pdf engines: marker, unstructured, surya_layout "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pdf_data = parser.parse(pdf_path)  # By default, images will be generated in the ./figure folder, but this can be changed with parameters."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(pdf_data.text)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### PPT Parser"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you need to parse PPT or DOC files, you'll need to download the `libreoffice` dependency. The installation methods are as follows:\n",
    "\n",
    "#### Mac\n",
    "\n",
    "```bash\n",
    "brew install libreoffice\n",
    "```\n",
    "\n",
    "#### Linux\n",
    "\n",
    "```bash\n",
    "sudo apt-get install libreoffice\n",
    "```\n",
    "\n",
    "#### Windows\n",
    "\n",
    "Visit the official website [https://www.libreoffice.org/](https://www.libreoffice.org/) to download."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from wisup_e2m import PptParser\n",
    "\n",
    "ppt_path = \"./test.ppt\"\n",
    "parser = PptParser(engine=\"unstructured\") # pdf engines: unstructured"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ppt_data = parser.parse(ppt_path)  # By default, images will be generated in the ./figure folder, but this can be changed with parameters."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(ppt_data.text)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### DOCX Parser"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from wisup_e2m import DocxParser\n",
    "\n",
    "docx_path = \"./test.docx\"\n",
    "parser = DocxParser(engine=\"pandoc\") # pdf engines: xml, pandoc"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "docx_data = parser.parse(docx_path) # By default, images will be generated in the ./figure folder, but this can be changed with parameters."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(docx_data.text) # You can find images were inserted in the docx file in ![]() format."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Core Feature 2: Converter\n",
    "\n",
    "The purpose of the Converter is to transform text or images after successful parsing. Currently, it supports converting these formats into Markdown using various model engines.\n",
    "\n",
    "Currently supported engines are `litellm` and `zhipuai`, with more engines to be supported in the future.\n",
    "\n",
    "- To see the models supported by Litellm, visit:\n",
    "  - [https://docs.litellm.ai/docs/providers/](https://docs.litellm.ai/docs/providers/)\n",
    "- To see the models supported by Zhipuai, visit:\n",
    "  - [https://open.bigmodel.cn/dev/howuse/model](https://open.bigmodel.cn/dev/howuse/model)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Text Converter"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from wisup_e2m import TextConverter\n",
    "\n",
    "text_converter = TextConverter(\n",
    "    engine=\"litellm\",\n",
    "    api_key=\"<your api key>\",\n",
    "    model=\"deepseek/deepseek-chat\",\n",
    "    caching=True,\n",
    "    cache_type=\"disk-cache\",\n",
    ")\n",
    "\n",
    "raw_text = docx_data.text"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "✨ Currently, the Converter only supports the default strategy.  \n",
    "Under the `default` strategy, the model will first determine the **text type** and **text format** and then perform sequential conversion to ensure the continuity of the generated Markdown text."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "markdown_text = text_converter.convert(raw_text) #  strategy = \"default\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(markdown_text)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Image Converter\n",
    "\n",
    "The Image Converter (`ImageConverter`) uses multimodal large models for recognition and can work with the `surya_layout` engine of the `PdfParser` for more detailed image recognition.\n",
    "\n",
    "Layout recognition engines with the `layout` suffix do not generate text; their main function is to recognize the layout within images. They then extract images that cannot be converted into text and mark them accordingly."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "from wisup_e2m import PdfParser, ImageConverter\n",
    "\n",
    "work_dir = os.getcwd()\n",
    "image_dir = os.path.join(work_dir, \"figure\")\n",
    "\n",
    "test_surya_layout_pdf = \"./test_surya_layout_pdf.pdf\"\n",
    "\n",
    "# load parser\n",
    "pdf_parser = PdfParser(engine=\"surya_layout\")\n",
    "\n",
    "image_converter = ImageConverter(\n",
    "    engine=\"litellm\",\n",
    "    api_key=\"<you api key>\",\n",
    "    model=\"gpt-4o\",\n",
    "    base_url=\"<you base url>\",\n",
    "    caching=True,\n",
    "    cache_type=\"disk-cache\",\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# parse the pdf with layout analysis\n",
    "test_surya_layout_pdf_data = pdf_parser.parse(\n",
    "    test_surya_layout_pdf,\n",
    "    start_page=0,\n",
    "    end_page=20,\n",
    "    work_dir=work_dir,\n",
    "    image_dir=image_dir, # extracted images will be saved to this directory\n",
    "    relative_path=True, # wheather to save extracted images with relative path or absolute path\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "test_surya_layout_pdf_data.to_dict()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "test_surya_layout_pdf_data.to_dict().keys()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# let's check the layout of the pdf file\n",
    "# you can find the header part of the pdf file is hidden, and the image part is recognized correctly.\n",
    "from IPython.display import Image, display\n",
    "\n",
    "image_16 = test_surya_layout_pdf_data.images[16]\n",
    "print(image_16)\n",
    "\n",
    "display(Image(image_16))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Then use ImageConverter to convert the images to markdown format.\n",
    "\n",
    "image_text = image_converter.convert(\n",
    "    images = test_surya_layout_pdf_data.images,\n",
    "    attached_images_map= test_surya_layout_pdf_data.attached_images_map,\n",
    "    work_dir = work_dir, # images will be relative to work_dir, default is absolute path\n",
    "\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# save Lecture Notes in Artificial Intelligence\n",
    "with open(\"Lecture Notes in Artificial Intelligence.md\", \"w\") as f:\n",
    "    f.write(image_text)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "e2m",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.14"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
